Skip to content

Implemented Durable Graph Database Providers ( golem:graph WIT interfaces) #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 35 commits into
base: main
Choose a base branch
from

Conversation

harshtech123
Copy link
Contributor

/claim #22
/closes #22

note : the testing is undergoing with maintainer reviews , this is initial and core version of golem:graph with solid foundation , thank you !

@harshtech123
Copy link
Contributor Author

harshtech123 commented Jun 17, 2025

#Maintainers this pr contains unit tests that are ignored when env variables are not set , this test are the MOTIVATION for our upcoming golem integeration tests ! we will refactor this soon .

@harshtech123
Copy link
Contributor Author

update : testing has been completed for integration , wasm component based testing is undergoing !

@harshtech123
Copy link
Contributor Author

Screencast.from.2025-06-23.23-10-12.webm

arangodb

  • you have to run arangodb local-Instance , example

sudo docker run -d --name arangodb -e ARANGO_NO_AUTH=1 -p 8529:8529 -v arangodb_data:/var/lib/arangodb3 -v arangodb_apps:/var/lib/arangodb3-apps arangodb

@harshtech123
Copy link
Contributor Author

Screencast.from.2025-06-24.16-16-07.webm

Janusgraph

  • you have to run janusgraph local instance to test this , example
  1. we have to specify the database (cassandra) in this case , since janusgraph doesnt store data by default

sudo docker run -d --name cassandra --network janus-net -p 9042:9042 cassandra:3.11

  1. we also needs to run janus-net before the above step

sudo docker network create janus-net

  1. lastly we need to run janusgraph by specifying the database , channalizer ( since it servers websocket by default in this case it is web and http both) , maybe timeout .

sudo docker run -d --name janusgraph --network janus-net -e storage.backend=cassandra -e storage.hostname=cassandra -e gremlinserver.channelizer=org.apache.tinkerpop.gremlin.server.channel.WsAndHttpChannelizer -e GREMLIN_OPTS="-Dtx.max-commit-time=60000 -Dstorage.cassandra.read-consistency-level=ONE -Dstorage.cassandra.write-consistency-level=ONE" -p 8182:8182 janusgraph/janusgraph:latest

@harshtech123
Copy link
Contributor Author

Screencast.from.2025-06-24.22-41-12.webm

Neo4j

  • we also have to run this locally , you can use official docker command for neo4j by default it runs on http , also needs auth 🔢
    docker run \ --publish=7474:7474 --publish=7687:7687 \ --volume=$HOME/neo4j/data:/data \ neo4j

@harshtech123
Copy link
Contributor Author

Tip : i also specified the env variables for local , in golem.yaml you have to just uncomment it which ever provider your running .

also this all three are well tested under the real instances at each level , using ureq .

@noise64 noise64 self-requested a review July 23, 2025 08:12
golem-rust = { workspace = true }
log = { workspace = true }
serde_json = { workspace = true }
mime = "0.3.17"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove unused crates (in other Cargo.toml-s too)

use std::env;

/// Retrieves a configuration value from an environment variable, checking the provider_config first.
pub fn with_config_key(config: &ConnectionConfig, key: &str) -> Option<String> {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where is this used?

tx_url: &str,
statements: Value,
) -> Result<Value, GraphError> {
println!("[Neo4jApi] Cypher request: {statements}");
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use log functions instead of println-s and eprintln-s

use crate::golem::graph::types::ElementId;

/// Helper functions for creating specific error types
pub fn unsupported_operation<T>(message: &str) -> Result<T, GraphError> {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that these helpers are not used

use std::collections::HashMap;

/// Database-agnostic error mapper that can be specialized by providers
pub struct ErrorMapper {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also seem to be not used at all.

if response.status().is_success() {
Ok(response)
} else {
let status_code = response.status().as_u16();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error handling does not seems to match the Neo4j HTTP API error format, and does not extract or map the Neo4j specific error codes to the wit error model.
Also the error handling and mapping could be extracted, and would be nice to have at least a sample of the error payload in the generic case, if parsing as error failed (so one can see e.g. errors introduced by proxies)

}

/// HTTP status code to GraphError mapping
pub fn map_http_status(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are status codes are shared between different Graph APIs? Also, we should not rely on string searches for error mapping, we should rely on the specific error formats of the different providers, and those mappings should be part of the specific sub-crate.

}

/// Map ArangoDB-specific error code to GraphError
pub fn from_arangodb_error_code(error_code: i64, message: &str) -> GraphError {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This mapping should not be part of the common graph crate

@noise64
Copy link

noise64 commented Jul 24, 2025

A generic request for the provider tests: these should be added (e.g. as scripts), and should be part of the CI flow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement Durable Graph Database Provider Components for golem:graph WIT Interface
2 participants